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for a normal reward process 

Yi-Ching YacJE 

Academia Sinica 

Abstract: We consider the Gittins index for a normal distribution with un- 
known mean 9 and known variance where 8 has a normal prior. In addition 
to presenting some monotonicity properties of the Gittins index, we derive an 
approximation to the Gittins index by embedding the (discrete-time) normal 
setting into the continuous-time Wiener process setting in which the Gittins 
index is determined by the stopping boundary for an optimal stopping prob- 
lem. By an application of Chernoff 's continuity correction in optimal stopping, 
the approximation includes a correction term which accounts for the difference 
between the discrete and continuous-time stopping boundaries. Numerical re- 
sults are also given to assess the performance of this simple approximation. 



1. Introduction 

The classical multi-armed bandit problem is concerned with sequential design of 
adaptive sampling from k statistical populations with distribution functions F$ i , 
i = 1, . . . , k (k > 2) where Bi denotes the unknown parameter of the ith population. 
Specifically, the objective is to sample Y\, Y 2l . . . sequentially from the k populations 
so as to maximize the expected total discounted reward 

•>. = / i: - "• (y^'^)' h[( '' ° l> - 

where 7r is the prior distribution of (9±, . . . , 9k) and {-fj} is a (deterministic) discount 
sequence. The two most important types of discount sequence are uniform discount- 
ing with finite horizon N > (i.e. jj = 1 for j < N and -fj — for j > N) and 
geometric discounting with discount factor < (3 < 1 (i.e. -fj — /3 J ' _1 , j — 1, 2, . . .). 
While in general the optimal allocation rule can only be characterized via the 
dynamic programming equations which admit no general closed-form solutions, 
Gittins and Jones [l3( showed that under geometric discounting, when the prior 
distribution is a product measure dm(6\, . . . ,6^) = d7r 1 (0 1 ) x ••■ x dn^O]-), the 
optimal allocation rule is to sample at each stage from the population with the 
greatest (current) Gittins index. See also [ll| and pll ]. 

For a population with distribution function Fg and (current) prior distribution 
tt(9) of the unknown parameter 0, the Gittins index is defined by 

(1) A(7T,/3)=SUp 

«>1 
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where the supremum is taken over all (integer-valued) stopping times £ > 1 and 
X\, X2, ■ ■ ■ are (conditionally) iid with common distribution function Fg (given 9). 
Equivalently, \(tt, (3) is the infimum of the set of solutions A of the equation 



(2) -^- = sup E^E e 

1 - P £ > 



n=l ' 



In [12j], computational methods for calculating Gittins indices are described and 
applied to the normal, Bernoulli and negative exponential families with conjugate 
priors, which involve using backward induction to approximate the right-hand side 
of ([2]) with the supremum over £ > replaced by the supremum over < £ < N for 
some large horizon AT. For (3 close to 1, such computational methods become time 
consuming as a very large horizon N is required to yield an accurate approximation. 
Thus it will be useful to have accurate analytic approximations to Gittins indices 
especially for j3 close to 1. 

In this paper, we consider the normal case with unknown mean 9 and known vari- 
ance where 9 has a normal (conjugate) prior. Section [2] presents some monotonicity 
properties of the Gittins index. In particular, it is shown that the Gittins index is 
a nondecreasing function of the prior variance. In Section [31 a corrected diffusion 
approximation to the Gittins index is derived by embedding the (discrete-time) 
normal setting into the continuous-time Wiener process setting in which the Git- 
tins index is determined by the stopping boundary for an optimal stopping problem 
(first introduced in [2J). By an application of Chernoff's continuity correction, the 
approximation includes a correction term which accounts for the difference between 
the discrete and continuous-time stopping boundaries. Numerical results are also 
given to assess the performance of this simple approximation. To prepare for the 
derivations, Sections 13.11 and 13.21 briefly review, respectively, some properties of the 
Gittins index for a Wiener process and Chernoff's continuity correction in optimal 
stopping. 

The monograph of Gittins [l2| provides a comprehensive theory of dynamic 
allocation indices and explores the class of problems whose optimal solutions can be 



characterized by dynamic allocation indices. On the other hand, Lai 17[ and Chang 
and Lai Q have proposed simple index-type adaptive allocation rules that are 
asymptotically optimal in both the Bayes and frequentist senses either as N — *■ 00 
(under uniform discounting) or as (3 — > 1 (under geometric discounting). Brezzi 
and Lai have recently refined and modified these adaptive allocation rules in the 
presence of switching costs, while Hu and Wei [l5| have constructed asymptotically 
optimal adaptive allocation rules subject to the irreversibility constraint. Various 
applications of the theory of multi-armed bandits can be found in sequential clinical 
trials, market pricing, labor markets and search problems; see e.g. A mum i23- 



2. Some monotonicity properties of the Gittins index for a normal 
reward process 

In this section, we consider the case that Xi, X2, ■ ■ ■ are (conditionally) iid N(9, a 2 ), 
the unknown mean 9 has a prior ir = N(u, v) and the variance a 1 is known. The Git- 
tins index is denoted by A (u, v, cr 2 , (3). By location and scale equivariance properties 
(cf. [H, Section 6.4), 

(3) A (u, v,a 2 ,(3) =u + rX (0, v/r 2 , a 2 /r 2 , (3) 



for r > 0. 
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Lemma 1. The Gittins index A (u, v, cr 2 , (3) is nonincreasing in a 2 . 

Proof. We prove the lemma by a simple randomization argument. Fix < erf < erf. 
Let Xi,X 2 ,... be (conditionally) iid N(9,af) given 9, which is assumed to have 
a prior n — N(u,v). Let £1,62, ... be iid iV(0, cr| — cr 2 ) (independent of the Xj). 
Then X[ = X x + e x , X' 2 = X 2 + £2, • • • are (conditionally) iid N(9, of) given 9. For 
any stopping time £' > 1 with respect to the filtration T' generated by X[, X 2 , . . . , 
we have 

e C 

En Eg ^ f3 n 1 X' n = En Eg ^ /3™ + E^- Eg ^ /3™" 1 e„ 1{^'>„} 

n—1 n—1 n—1 

£' 

= En Eg ^ Z?"" 1 ^ . 

n=l 

Since every stopping time £' with respect to T' may be viewed as a randomized 
stopping time with respect to T (the filtration generated by X x , X 2 , ■ ■ ■), it follows 
that 

X(u, v, alP) = sup EnEgf^P^X^) I E^Egf^^ 



n—1 / 1 x ri— 1 

.up E- E„ ( V 3 n A X n ) I En Eg (V 3 n 

2 



completing the proof. □ 

Theorem 1. A (0, v, cr 2 , f3)/^/v is nondecreasing in v. 

Proof. For fixed < v 2 < v\ , it follows from and Lemma Q] that 



A(0, V 2 , CT 2 ,/3) = y/v 2 /vi A(0, Ui, CT 2 U1/U2, /3) 



< V«2M A(0, m, CT 2 , 0), 

completing the proof. □ 

Corollary 1. A(w, v, cr 2 , /3) = it + A(0, u, cr 2 , /3) is nondecreasing in u and v. 

Remark 1. For the Wiener process setting, Bather Q proved a result analogous 
to Theorem [I] (see © and © below). 

Remark 2. For a normal two-armed bandit in which the means of arms 1 and 
2 have independent normal priors N(ui,Vi) and N(u 2l v 2 ) and their variances are 
known and equal, it follows from Corollary [1] that under geometric discounting, 
it is optimal to pull arm 1 initially if u\ > u 2 and v\ > v 2 . It seems natural to 
conjecture that the same also holds under uniform discounting. Note that Berry [j| 
made a similar conjecture regarding a Bernoulli two- armed bandit, which has not 
been resolved (cf. [4|, Section 7.3). 

Remark 3. Along the lines of the proof of Theorem [IJ it can be readily shown 
that 

A(0, wi, ct 2 , 0)/y/vi > A(0, v 2 , o~ 2 , 13)/^ 
if vi > v 2 and v\ja\ > v 2 /a 2 . Note that for a normal distribution N(9,o- 2 ) where 
9 has a normal prior N(0, v), vja 2 may be referred to as the signal-to- noise ratio 
since v is the second moment of the "signal" 9. 
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3. Corrected diffusion approximation to the Gittins index for a normal 
reward process 

In Section 3.3, we derive an approximation to the Gittins index for a normal distri- 
bution whose mean is assumed to have a normal prior. To prepare for the deriva- 
tions, we briefly review, in Sections 3.1 and 3.2, some properties of the Gittins index 
for a Wiener process and Chernoff 's continuity correction in optimal stopping. 



3. 1 . Properties of the Gittins index for a Wiener process 

Bather Q showed that for a Wiener process {W{t),t > 0} with drift coefficient 
9 which has a normal prior N(uo,Vo), the Gittins index A*(uo,«o,c) can be de- 
termined by the solution to an optimal stopping problem (to be described below) 
where c > denotes the discount rate in continuous time (see also Q and Section 
6.6 of 12]). Here X*(u 0l v 0l c) is defined as the infimum of the set of solutions A of 
the equation (cf. ([2]) ) 



(4) 



e dt = sup E n Eg 

T>0 



sup Ek 

T>0 

SUp Ek 

T>0 

sup E v 

T>0 



l dW{t) + A 



dt + A 



'dt 



'dt 



u(t) e~ ct dt + A 



'dt 



u 



A e" 



whcrc the supremum is taken over all (real-valued) stopping times r > 0, tt 
N(uq,vo) is the prior distribution of 9, and u(t) is the posterior mean of 9, i.e. 



(5) 



u(t) = E„ 9 | W(s), 0<s<t 



«o x «o + W(t) 



+ t 



The last equality in (HJ) follows from integration by parts along with the fact that a 
simple change of time transforms u into standard Brownian motion, cf. Y(v) below. 
Define 

v = v(t) — (vq 1 + 1)^ 1 (the posterior variance), s — v/c, 
Y(v) = u a - u(t), and Z(s) = Y(cs)/y/c. 

It can be readily shown that {Y(v), < v < vo} is standard Brownian motion 
(Y(vq) = 0) in the — v scale and {Z(s), < s < s } (s — v /c) is standard 
Brownian motion ( Z(sq) = ) in the — s scale. Letting zq — (A — uq) / yfc, it follows 
that (j4|) is equivalent to 



(G) 



z e 



-l/s _ 



sup E 

0<S<s o 



{Z(S) + z a } e" 1 ' 8 



in the sense that A is a solution of (|4|) if and only if zq — (A — uq)/^/c is a solution 
of ([6]) , where the supremum on the right-hand side of ((6]) is taken over all stopping 
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times < S < so (in the — s scale). It is more convenient to remove the restriction 
of Z(sq) = and rewrite (J6]) as 



-l/«o - 



sup E 

0<S<s 



Z(S) e-^ s | Z(s ) = z 



For the optimal stopping problem with payoff function g(z,s) = ze^ 1 ' 5 on the 
right-hand side of jHI]) , it is easily shown that the continuation region is of the form 
{(z, s) : z < b(s)} where b(s) > is the optimal stopping boundary. Since zq is a 
solution of (HQ if and only if (zq, sq) is in the stopping region (i.e. zq > 6(sq)), it 
follows that A*(w , vq,c), the infimum of the set of solutions A of the equation (|4]), 
satisfies b(so) — (\*(uq, Vq, c) — uq) / yfc , i.e. 

(7) A* (u , w , c) = u + Vc b(s ) = uq + Vc b(v /c) . 

Bather showed that 



(8) 



b(s)/y/s is a nondccreasing function of s . 



(9) 



&(s) < s/v^ for all s > , and lim b(s)/s = 1/V2 . 

s— >0 



while Chang and Lai [6| derived the asymptotic expansion as s — > oo 

1/2 



(10) 



&(*) 



2s 



Based on (|8])- (|T0]) together with extensive numerical work (involving corrected 
Bernoulli random walk approximations for Brownian motion), Brezzi and Lai [f| 
have suggested the following closed-form approximation 'I'(s) to b(s)/y r s 



(ID *£) 



9(a) 



Vs/2 

0.49- 0.11s" 1 / 2 
0.63- 0.26 s- 1 / 2 
0.77 -0.58 s- 1 / 2 



for s < 0.2 , 
for 0.2 < s < 1 
for 1 < s < 5 , 
for 5 < s < 15 , 



r i 1/2 

|2 log s — log logs — log 167r| for s > 15 



3.2. Chernoff's continuity correction in optimal stopping 

In his pioneering work, Chernoff studied the relationship between the solutions of 
the discrete and continuous-time versions of the problem of testing sequentially the 
sign of the mean of a normal distribution. His result may be stated more generally 
as follows. Let {B(t)} be standard Brownian motion and let g(x,t) be a smooth 
payoff function for t < T (horizon) for which the continuation region is of the form 
{(x, t) : x < b(t)}. Consider a constrained optimal stopping problem where stopping 
is permitted only at nS, n = 1,2,... where S is a given (small) positive number. 
Suppose that there exist stopping boundary points bs(nS), n = 1,2,... such that 
starting from B(no6) = xq for any given no and xq, the optimal stopping rule is to 
stop at the first n > uq at which B(nS) > bs(nS). So bs(nS) (or b(t), resp.) is the 



Some results on the Gittins index for a normal reward process 



289 



discrete-time (or continuous- time, resp.) stopping boundary for the constrained (or 
unconstrained, resp.) optimal stopping problem. Then for fixed t < T, we have 

(12) b s (t) = b(t) -pVS + o(VS) as <5^0, 

where b 5 (t) = b s ([t/5]5), p = ES 2 + /2ES T+ » 0.583, r+ = inf{n : S n > 0}, 

S n = Xx H h X n , and the X t are iid N(Q, 1). 

Chernoff [7| derived l]12p by relating the original problem to an associated stop- 
ping problem in which there is a horizon at t = and the payoff function is g(x, t) = 
—t + x 2 l{a;<o.i = o}i t <Q. For the associated stopping problem, stopping is permit- 
ted at 0,-1, —2, . . . , and there exist stopping boundary points b-i > 6-2 > • • • 
such that starting from (xo,Tio) with hq < 0, the optimal stopping rule is to stop 
at the first no < n < at which 

x +Xx-\ hX„_„ >6„ (6 = -00). 

Chernoff 0] and subsequently Chernoff and Petkau 0] and Hogan 14 1 showed that 

lim b n = -ES*/2ES T , 

n — > — 00 + 

for normal, Bernoulli and general X (with finite fourth moment), respectively. Re- 
cently, under mild growth conditions on g, Lai, Yao and AitSahlia [18| have proved 
(|12|) when the Brownian motion process is replaced by a general random walk in 
the constrained problem. 



3. 3. Approximating the Gittins index for a normal reward process 



In this subsection, we consider the case that X\ , Xi , . . . are (conditionally) iid 
N(d, o~ 2 ) and the unknown mean 9 has a prior n = N(uq, vq). Without loss of gen- 
erality, we assume a 2 = 1. For notational simplicity, the Gittins index A(mo, vq, 1, (3) 
will be abbreviated to A(tto, vq, 0). Recall that A(uo, vq, 0) is the infimum of the set 
of solutions A of the equation As in Section [37X1 let {W(t),t > 0} be a Wiener 
process with drift coefficient 6 which has a normal prior N(uq,vq). Noting that 
{X\,X%, . . ) and (W(l), W(2) — W(l), . . .) have the same joint distribution, we can 
rewrite fl2J) as 

sup E v Eg 



A 



1-/3 



= sup E n 



J^P^fw^) -W(n-1] 

71=1 ^ 



A 



L n=l 



(3 n - x u{n-l) + ft 



A 



sup E n 
— !— sup E„ 



1 



1-/3 

uo- (u(0-A)e- c « 



where is given in ([5]), c = — log/3 and the third equality follows since 



E 



1-/3 



n-l 
C 



u(t) e^ ct l {e >„ } I W(s), < s < n - 1 



1 a !{$>»! / w(n-l)e ct dt = (3 n 1 u(n - 1)1 {C >„ } 

1 — P in-l 
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With the notation introduced in Section [37X1 we can further rewrite ((2J as 



A — «o = sup E 

V£{v /(l+v n) ,n=0,l,... } 



(A-uo + r(V0) 



e _ c y-i +cl ,- 1 



where the supremum is taken over all stopping times V taking values in { Vo/(l + 
von), n = 0,1,...}. In terms of Brownian motion Z(s) in the —s scale, (J2J is 
equivalent to 



(13) z e- 1/so = sup E 

s e {c-^o/ti+fo") , ti=o,i,...} 



Z{S)e- 1 / s | Z(s ) = z 



where z$ = (A — uo)/y/c, sq = vq/c and the supremum is taken over all stopping 
times S taking values in {c i>o/(l + Von) , n = 0, 1, .. .}. 

For the constrained optimal stopping problem on the right-hand side of (|13|) . 
there exist optimal stopping boundary points b Vo (c vo / (1+Vori)) , n = 0, 1, . . . such 
that the optimal stopping rule is to stop at the first n at which Z(c~ 1 vq/ (I+vqu)) > 
&u (c~ 1 fo/(l + «o ra ))- So 6„ (vo/c) is the infimum of the set of solutions zq of the 
equation (|13p . It then follows that 



(14) A(w ,vo,(9) = u + Vcb Vo (v /c) . 

Since in the constrained optimal stopping problem the permissible stopping time 
points c -1 i>o/(l + von), n = 0, 1, 2, . . . are not equally spaced, there is no rigorous 
justification for applying (112|) to relate the discrete and continuous-time stopping 
boundaries b vo (t) and b(t) for the constrained and unconstrained problems. How- 
ever, it can be argued heuristically that (|12|) applies when the spacing between 
many successive permissible stopping time points is approximately constant ( cf. 
bottom of page 47 in [l3|). Thus we arrive at the following approximation 

(15) b V0 (v /c) « b(v /c) - 0.583%/£ 
where 

§ _ c~ 1 v c^vq c~ 1 v$ 



1 + VQ ■ 1 + VQ-l l + VQ 

provided that vq /c is bounded away from (the horizon of the optimal stopping 
problem) and 5 ps i + J° n ~ i+^ (n+i) ^ or man y (small to moderate) n's. That is, we 
expect the approximation (115|) to be reasonably good if vo is small and vo /c is not 
too close to 0. It follows from Q3J), (HJl, (HI and (HU that 

A(w > vo, /?) Ps uo + \/c b(vo/c) — 0.583 w / \/l + 



(17) ps u + V^o *(«o/c) - 0.583 v /vT+^. 

Note that the continuation region for the constrained problem must be con- 
tained in the continuation region for the unconstrained problem, so that b Vo (vq / c) < 
b(v /c). Thus the uncorrected diffusion approximation u + y/cb(v /c) overestimates 
X(uo,vq, (3) = uq + ^/cb vo (vo/c), which is recorded in the following theorem. 

Theorem 2. X(u , v , (3) < u + y/c b(v /c) where c — log /3 _1 . 
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A related upper bound for \{uq, vq,0) is given in Theorem 6.28 of Gittins [lz 
which states, in our notation, that 



(18) X(u , v , (3)<u a + y/l-p 6(«o/(l - /?)) . 

Since b(s)/yfs is nondecreasing in s by © and since c = log/? -1 > 1 — 0, we have 



b(vo/c) < VW? &(«o/(l - /?)) . 

so that the upper bound given in Theorem [2] is sharper than (fP8|) . 

In the approximation (|15[) . the correction term 0.583V5 with £ given in (|16[) 
appears to be a little too large since the spacing between successive permissible 
stopping time points 1 + J Q ° n — 1+ c Vq ^° +1 -j is strictly less than § for n > 1. To com- 
pensate for this overcorrection, we propose (in view of (J9j) ) to replace b(vo/c) by 
(vq/c)/>/2 in (|15p . resulting in the following simple approximation 



(19) 



\(uo,vo,0) ~ u + vq/V2c— 0.583 vo/y/l + vo 



Note that agrees with (JT7J) for w /c < 0.2 in view of (fTTj) . 

In his Table 1, Gittins [lj tabulates n(l - /3) 1 / 2 A(0,n _1 ,/3) for various values 
of n and /?. Our Table Q] compares n(l — /3) 1 / 2 A( 0, /3 ) with the corrected and 
uncorrected approximations ( based on (jTTJ) and (fT^) ) 



(CA) n(l-j9)3 



0.583 n" 



= (1-/3)1 



n 1 
0.583 



Vl + n~ 

(UA ) n(l-/?)ll$(i)=(l-^^) 



(CA,) (1 "^L^ VT 

(UA') V(l-/9)/(2c). 



1 



0.583 



Remark 4. As explained earlier, the uncorrected approximations have positive bias 
due to overestimation. The corrected approximations are reasonably accurate for 
moderate to large n and for large (3. For moderate n, (CA) (or (CA'), resp.) tends 
to underestimate (or overestimate, resp.) n(l — P) 1 ^ 2 A(0, n _1 , /?). This observation 
naturally leads to approximating n(l — /3) 1 / 2 A(0, /3) by the average of (CA) 
and (CA'), which is also included in Table CD Overall, [ (CA) + (CA')]/2 has the 
best performance, while (CA') is better than (CA) except for small n and large j3. 

Remark 5. Table 1 of Gittins [H suggests that n(l - /3) 1 / 2 A(0, /3) is in- 
creasing in n. For /3 = 0.5,0.6,0.7,0.8,0.9,0.95, Gittins has numerically estimated 
linin^oo n(l — /3) 1 / 2 A(0, n" 1 , /?). These numbers are compared in Tabled with the 
limits (1 - P) 1 / 2 [{2c)- 1 / 2 - 0.583] (or (1 - /3) 1 / 2 /(2c) 1 / 2 , resp.) obtained from the 
corrected approximations (CA) and (CA') (or uncorrected approximations (UA) 
and (UA'), resp.) as n — > oo. It should be noted that the heuristic justification for 
the corrected approximations requires vq/c — l/(nc) not to be very close to 0. 
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Table 1 

Gittins indices and approximations 
(0 = 0.5, 0.7, 0.9, 0.95, 0.99, 0.995) 



10 50 100 500 1000 



0=0.5 

n(l- /3) 1 / 2 X&n- 1 ,/]) 

[(CA) + (CA')]/2 

(CA) 

(CA') 

(UA) 

(UA') 

0=0.7 

[(CA) + (CA')]/2 

(CA) 

(CA') 

(UA) 

(UA') 

0=0.9 

n(l — /3) 1//2 A(0, n _1 , /3) 

[(CA) + (CA')]/2 

(CA) 

(CA') 

(UA) 

(UA') 

0=0.95 

n(l - 0)^(0, n.-\0) 

[(CA) + (CA')]/2 

(CA) 

(CA') 

(UA) 

(UA') 

0=0.99 

n{\- pf^X^n- 1 ^) 

[(CA) + (CA')]/2 

(CA) 

(CA') 

(UA) 

(UA') 

0=0.995 

n{l- PY^X^n- 1 ,^) 

[(CA) + (CA')]/2 

(CA) 

(CA') 

(UA) 

(UA') 






.211 


0. 


.224 


0. 


226 


0. 


227 


0. 


.227 


0. 


.208 


0. 


.192 


0. 


.190 


0. 


.189 


0. 


.189 


0. 


.208 


0. 


.192 


0. 


190 


0. 


.189 


0. 


.189 





.208 





.192 


0. 


.190 


0. 


189 


0. 


.189 





.601 





.601 


0. 


601 


0. 


601 


0. 


.601 


0. 


.601 


0. 


.601 


0. 


601 


0. 


601 





.601 





.311 





.337 


0. 


341 


0. 


344 


0. 


.345 


0. 


.264 


0. 


332 


0. 


331 


0. 


329 





.329 


0. 


.184 


0. 


.332 


0. 


331 


0. 


329 





.329 





.344 





.332 


0. 


331 


0. 


329 


0. 


.329 


0. 


.489 


0. 


.648 


0. 


648 


0. 


648 





.648 


0. 


.648 


0. 


.648 


0. 


648 


0. 


648 


0. 


.648 





.415 


0. 


.480 


0. 


493 


0. 


504 


0. 


.506 


0. 


.357 


0. 


.506 


0. 


505 


0. 


505 





.505 


0. 


.201 


0. 


.506 


0. 


505 


0. 


505 





.505 





.513 





.506 


0. 


505 


0. 


505 


0. 


.505 





.377 





.689 


0. 


689 


0. 


689 


0. 


.689 


0. 


.689 


0. 


.689 


0. 


689 


0. 


689 





.689 





.425 


0. 


.519 


0. 


.540 


0. 


562 


0. 


.566 


0. 


.382 


0. 


.468 


0. 


568 


0. 


568 





.568 


0. 


.190 


0. 


.367 


0. 


568 


0. 


568 





.568 





.574 





.569 


0. 


568 


0. 


568 


0. 


.568 





.314 





.496 


0. 


698 


0. 


698 


0. 


.698 


0. 


.698 


0. 


.698 


0. 


698 


0. 


698 





.698 





.353 


0. 


.499 


0. 


549 


0. 


618 


0. 


.633 


0. 


.390 


0. 


.453 


0. 


485 


0. 


.647 





.647 


0. 


.130 


0. 


.257 


0. 


322 


0. 


647 





.647 





.650 





.648 


0. 


.647 


0. 


.647 


0. 


.647 





.185 





.315 


0. 


380 


0. 


705 


0. 


.705 


0. 


.705 


0. 


.705 


0. 


.705 


0. 


705 


0. 


.705 





.304 


0. 


.457 


0. 


516 


0. 


.614 


0. 


.638 


0. 


.424 


0. 


.437 


0. 


470 


0. 


562 


0. 


.665 


0. 


.181 


0. 


.209 


0. 


274 


0. 


458 


0. 


.665 





.667 





.665 


0. 


665 


0. 


665 


0. 


.665 





.221 





.250 


0. 


315 


0. 


499 


0. 


.706 


0. 


.706 


0. 


.706 


0. 


706 


0. 


706 





.706 



Remark 6. Brezzi and Lai [5j] have proposed a simple approximation to Gittins 
indices for general distributions which is justified by making use of the functional 
central limit theorem as /3 — ► 1. For Bernoulli distributions (with beta conjugate 
priors), their approximation provides fairly accurate results. When applied to nor- 
mal distributions, their approximation reduces to the uncorrected approximation 
(UA). It will be of great interest to see whether and how Chernoff's continuity 
correction can apply to approximate Gittins indices for nonnormal distributions. 
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Table 2 





The limits of Gittins indices 


and approximations 




p 


li mjwoo n(l -/3) 1 / 2 A(0,n-\/3) 


(CA) and (CA') 


(UA) and (UA') 


0.5 


0.227 


0.189 


0.601 


0.6 


0.283 


0.257 


0.626 


0.7 


0.345 


0.329 


0.648 


0.8 


0.417 


0.409 


0.669 


0.9 


0.509 


0.505 


0.689 


0.95 


0.583 


0.568 


0.698 
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